Most companies and use cases do not care about PhD-level capabilities. They just want an AI model to depend on with absolute reliability, for the simple tasks in life.

Imagine a highly competent, dependable, reliable assistant, who does all your chores–instead of the caffeine-overloaded genius, who sometimes works when it “feels like it”--but otherwise causes more harm than good half the time.

That's what we just built with the Featherless.ai Action-R1 model & agent, which achieved SotA (State-of-the-Art) in the REAL (Realistic Evaluations for Agents Leaderboard) benchmark.

Our AI agent (test result at link) achieved a 65% success rate on the REAL benchmark. This makes it nearly 50% better than Anthropic computer use (result link), the next best model & framework by a major lab, at 42%.

What is the REAL benchmark?

What is interesting about the REAL benchmark is that it tests over 110 practical real-world tasks. These are not university-level knowledge tasks, but real-world chores that reflect what people do online every day, like booking flights, organizing and replying emails, and shopping for groceries. Basically, office desk work.

These are tested using a controlled test replica, which mirrors 11 major websites, including Airbnb, Amazon, Gmail, LinkedIn, and Uber.

The paper for the benchmark can be found here.

Why does reliability matter?

Frontier AI agents have been plateauing at around 43% overall task completion. We hit 65%, leapfrogging it by 22 percentage points.

More importantly, on certain sites, such as the Omnizon (aka Amazon clone), our agent reached a 100% success rate, while the next best managed 60%.

And the 99%+ distinction matters.

Because you can now rely on this model and agent for tasks in this domain.

If a model is only at 50% success rate at a category of task, you spend more time and energy “babysitting” the models. In several cases, the model will require human intervention which can take more bandwidth than the task itself.

Raising reliability means that you can hand off tasks to the AI, eliminating the frustration many workers face with AI in the enterprise.

For the first time, we have built an AI agent you can rely on for tasks within a handful of platforms with 99%+ success. We will eventually expand to 99%+ reliability for all web platforms.

How we did it

We partnered with the team at UI-licious, who automate end-to-end testing for their customers with AI at scale, building on their experience and knowledge in UI test automation and their proprietary PetaByte scale dataset which they have built up over the years on UI testing.

Together, we co-built a specialized AI action model and agent harness. This helps the AI understand instructions and how to navigate complex and dynamic UIs.

As a bonus, these AI agents are not just navigating and completing tasks. They are finding and issuing bug reports, which we will be following up with the eval maintainers on 😉

The bigger picture: attacking multi-billion dollar markets

The $3B+ Robotic Process Automation (RPA) market and the $20B+ UI testing market are dominated by brittle tools that break easily and require constant maintenance, where AI agents achieve about 60% reliability.

Today’s RPA tools like UIPath operate like glorified macros, breaking when user interfaces change or when variability is introduced. Businesses spend millions maintaining these systems, fixing broken RPA scripts.

Agentic AI is different. It learns. It adapts. It interacts with interfaces the way humans do: observing, interpreting, and acting based on context. It allows automation over dynamic interfaces.

And we’re just getting started.

As we scale our AI research in the open source space, we will be working alongside industry partners to build reliable AI Agents and models for their own industry specific use, with their own proprietary datasets.

So we will be increasing reliability, not just in automated UI testing, but in all future domains as well, with industry partners from accounting to legal.

Raising reliability means unlocking the 90% of AI projects that fail to enter production within enterprises, as the AI was just “not reliable enough” for day-to-day office tasks.

Raising reliability means bringing into production your AI agent into the market.

If you are interested in partnering with us, and making your AI more SoTA-reliable, please reach out to us.

How can we try this new action model & agent?

Look out for a follow-up launch announcement with our design partners at UI-licious.

This AI action model and agent will be co-launched on both platforms.

For early access, you can sign up here:
https://forms.gle/wxwQ2z12xf1KzFvPA

Priority access will be given to portfolio companies among our investors and users.

"AI has the power to transform industries and empower individuals around the globe. However, accessing and utilizing a diverse range of models can be cost-prohibitive and logistically challenging for many users, particularly those in emerging markets, where there's no shortage of demand for AI solutions," notes our Founder and CEO Eugene Cheah. "Featherless.ai is changing the game by providing an affordable and scalable solution for using open source AI models in production. I don't want a future where AI is controlled by the few. I want to empower individuals globally."

What We Offer: The World's Largest Collection of Open-Source Models

At Featherless.ai, we provide instant and affordable access to over 4,000 open-source AI models - including popular options like DeepSeek and LLama - and we're continuously onboarding new models every week. Our flat capacity pricing model ensures cost predictability and scalability, allowing businesses to dynamically scale AI usage without worrying about unexpected charges or rate limits.

Instead of dealing with infrastructure headaches, our users can focus their energies on what matters: running experiments to develop, test, and fine-tune their models. With this new funding, we're planning to support all major AI modalities — including embeddings, vision, and speech.

For casual AI users, our serverless platform offers a low entry point of just $10 per month, providing access to a wide range of the latest open-source models without the need for expensive high-end GPUs. For businesses and enterprises, we deliver scale and significant cost efficiencies through dynamic AI workload scaling and flat capacity pricing.

Our Breakthrough Technology

With our new funding, we're advancing research and development into next-generation AI architectures that can dramatically lower inference costs, making AI deployment feasible on lower cost hardware rather than requiring high-end GPUs.

Our architecture enables seamless support for over 100 languages without any performance degradation. This breakthrough removes a major barrier to global AI deployment, where existing systems struggle to maintain consistency across multiple languages.

One of our proudest innovations is our proprietary hot-swapping technology, which allows AI models to be switched in under 5 seconds, compared to the 30 minutes typically required to load a new AI model onto a standard GPU. This innovation optimizes GPU utilization, eliminates costly downtime, and dramatically reduces operational costs.

We're also actively involved in the development of the open-source RWKV foundation model project, the first AI model under the Linux Foundation, which has been deployed on Windows to billions of devices1. This effort is part of our broader commitment to develop better and more accessible AI models across languages, and reduce the reliance on closed-source models from major tech companies.

What Our Investors Are Saying

"The biggest hurdle to scaling AI is compute cost. Featherless.ai tackles this head-on by providing the trifecta of efficient, effective and affordable AI deployment to businesses globally. Our portfolio companies are avid users of Featherless.ai, and we look forward to expanding that across all the regions we invest in," shares Vishal Harnal, Managing Partner at 500 Global.

Yuichiro Hikosaka, Principal at Airbus Ventures, explains: "Our research into foundational model development revealed that while much of the AI industry has focused on scaling existing transformer models, fundamental inefficiencies in the architecture have remained unaddressed. As most foundational models struggle to maintain performance across multiple languages, creating accessibility barriers in global markets, these structural inefficiencies have threatened to stifle competition and innovation across the broader global ecosystem. Now enter Featherless.ai - a serverless platform that empowers users to run, test, and fine-tune the latest models seamlessly and affordably. Featherless.ai is ready and primed to address a global market."

"We see Large Language Models being experimented and integrated into applications everywhere, Featherless.ai plays a unique role in enabling organizations to test, integrate and run any and every open source LLM easily and cost-effectively. Their innovative approach and focus on serving diverse markets make them a standout in this rapidly growing space," says Chee-We Ng, Managing Partner at Oakseed Ventures.

Our Team

Featherless.ai is co-founded by Eugene Cheah (CEO), Harrison Vanderbyl (CTO), and Wesley George (COO), who bring combined experience of 30 years building software and leading engineering teams.

Join us!

Featherless.ai is hiring! If you're passionate about making AI open and accessible, send your cover letter to hello@featherless.ai. For more information about our platform and services, visit www.featherless.ai.

Together, we can democratize access to AI and ensure that these powerful technologies benefit everyone, not just those with unlimited resources.

AI development continues to advance through diverse approaches to model design and optimization. DeepSeek-R1, with its impressive 671B parameters, has established itself as one of the most capable reasoning-focused models on the market. Its remarkable capabilities have set new benchmarks for what models in this space can achieve.

Meanwhile, efficiency and adaptability continue opening new frontiers, and this is where QwQ-32B, Qwen's latest release, makes its mark.

QwQ-32B: Efficient Reasoning Power

QwQ-32B delivers high-level reasoning, problem-solving, and strong coding/math capabilities in a lightweight package. With just 32B parameters, early benchmarks show impressive performance, making it an attractive option for those looking for strong reasoning capabilities in a more efficient format.

With AI applications diversifying, the demand for models that deliver excellent performance with a smaller resource footprint continues to grow. QwQ-32B exemplifies how well-optimized models can achieve remarkable results.

The Evolution of Reasoning Models

The AI field is evolving rapidly. While early models focused heavily on generative fluency and knowledge retrieval, today's most exciting breakthroughs are in models that can reason, plan, and solve complex problems. DeepSeek-R1 has been instrumental in this evolution, demonstrating the power of advanced reasoning capabilities.

Now, Qwen is expanding possibilities further, showing that reasoning power can be delivered in different formats to meet diverse needs.

How Does QwQ-32B Perform?

Testing indicates that QwQ-32B:

• Excels in logical reasoning tasks with impressive structured problem-solving

• Performs well in math and coding, key benchmarks for reasoning capability

• Offers strong efficiency, delivering high performance with reduced compute requirements

For users seeking high-quality reasoning in an efficient package, QwQ-32B presents an exciting option.

What This Means For You

With both QwQ-32B and DeepSeek-R1 available on Featherless.ai, you now have multiple excellent options for advanced reasoning capabilities. Our ongoing optimization efforts ensure that you'll benefit from continuous improvements in both performance and functionality.

We're committed to making advanced open AI models accessible and practical for everyone. Each model in our lineup offers unique advantages to suit different use cases and requirements.

Try Our Models and Share Your Thoughts

QwQ-32B is now available alongside DeepSeek-R1, and we want to hear about your experiences with both models. Each excels in reasoning tasks while offering different profiles in terms of scale and efficiency.

Leave a review on the Featherless.ai model page: QwQ-32B

Have questions about integrating these models into your workflow? Reach out to us on Discord or check our documentation for implementation guidelines and best practices.

Over the past year, Recursal.AI has served us well as a name and identity for our development team and our various projects. It has been a banner under which we’ve worked to push the boundaries in AI.

However overtime, we are now more commonly identified by our platform Featherless.AI, which has grown to be the biggest provider of HuggingFace LLMs, striving to make them more accessible, and free from the constraints of closed ecosystems.

It has become the identity most closely associated with our work, our vision, and the principles that drive us forward.

So today, we’re making it official: Recursal.AI is now Featherless.AI. This change isn’t just about branding—it’s a reaffirmation of our commitment to the usage of open-source AI.

We remain dedicated to bringing powerful language models to the wider world for the benefit of the many.

Our mission remains the same: to develop and support AI tools that are transparent, accessible, and unshackled from proprietary control. Here’s to the next chapter—under a name that truly reflects the community and future we’re building together.

— The Featherless.AI Team

This is a game with elements that are quite challenging for “traditional” software development that are remarkably easy with an LLM. If you want to try what we built before reading further, you can do so here!

The Challenges

Building a game requires solving lots of problems around input, output, UI, visuals, latency, the list goes on. We’ll be focussing on the content related challenges that are not easy for the “rules-based” nature of typical software.

When we think about building Family Feud - or Featherless Feud as we call this version - we hit two problems:

1. question and answer generation

2. fuzzy answer matching

i.e. how do come up with quiz questions, and when given answers that aren’t exactly on the list, how do we match them?

For the actual show, the question bank is built with the help of surveys of the general public, meanwhile identifying variation of official answers are done by show staff live, a natural language task which is simple for humans, but challenging for software.

In both cases, we’ll be prompting an LLM to generate a certain kind of text and interpreting the results, but beyond just prompting, how we run the inference also matters. Let’s dive in.

Part 1 - Question/Answers Generation

To generate questions, we might prompt an LLM like so

You are the producer of the game show Family Feud.

Your job is to devise a question and a list of common answers to this question. For each answer, output a number between 0 and 100 which is how common you think the answer would be when given by a member of the general public. The sum of the scores should not exceed 100.

Now that content has structure (e.g. question versus answers) that we need to know about so that the various pieces can be handled appropriately by the different parts of our app.

The LLM “knows” about the relationship between the questions and answers in an output, so we can ask it to include the annotations in the output. We really could ask the LLM to output any kind of structured format but the most common format to request output in is JSON.

As for getting a model to formulate it’s output as JSON there are typically two parts:

1. prompting for the answer to be in JSON with a description of desired structure

2. requesting JSON from your inference API call

For 1., you might update your prompt as follows

You are the producer of the game show Family Feud.

Your job is to devise a question and a list of common answers to this question. For each answer, output a number between 0 and 100 which is how common you think the answer would be when given by a member of the general public. The sum of the scores should not exceed 100.

Be very concise, present only the question and answers, and don't add anything else.

Output valid JSON with the following format: two keys, `question`, which should be a string, and `answers` should be an array of objects. Each object in the `answers` array should have two keys, `answer` and `score`, each of which should be a string.

e.g.
```json
{
  "question": "Name something that might be wobbly",
  "answers": [
    { "answer": "Furniture" , score: 64 },
    { "answer": "Person/A Drunk", score: 15 },
    { "answer": "Spinning toy/top", score: 5 },
    { "answer": "Shopping Cart", score: 3 }
  ]
}
```

The sum of the scores should not exceed 100.

Being prescriptive in structure is important; if the output description isn’t clear enough to the model, you’ll get variation in structure and keys used, which likely cause an exception when the part of your software operating on the JSON output tries to separate the question from the answers.

Though even with an explicit description of output structure, the model to output syntactically valid JSON is it is unlikely that if you prompting only will have your output will be valid JSON. Here’s a sample of the output from above:

Here is a question with a list of answers and scores:

{
  "question": "Name something you might find in a purse or wallet",
  "answers": [
    { "answer": "Money", score: 70 },
    { "answer": "Phone", score: 20 },
    { "answer": "Credit Card", score: 5 },
    { "answer": "Candy", score: 2 },
    { "answer": "Makeup", score: 3 }
  ]
}

Pretty good right? It came up with a question, and in the format as requested (and needed for the game). It also outputted the `score` values as Numbers even though I asked, in the prompt, for their input to be strings! The model overrode my instructions, but for the better. However despite the request to give nothing but the output, there still is some pre-amble which requires it’s own intelligence to parse. How do we get rid of this?

The other half the equation is an inference technique known as “guided decoding” in it’s most general form, but known as JSON mode. Here we guarantee the output is syntactically valid JSON by manipulating token probabilities during sampling to prevent syntactically invalid JSON from appearing (e.g. the string {‘‘} is not valid JSON).

Of course these two modes are in tension with another - if you don’t ask for JSON in your prompt (or specify structure clearly enough), and put on JSON mode, it’s possible that your output bears little relationship to your input. In an early version of the prompt above, I got a number of questions.

How you request JSON mode varies by your API provider (this is a specific type of guided decoding which is a big topic), but many (most?) providers that have an API will accept the same format to the request as OpenAI, which is to include a

"request_format": { "type": "json_object" }

in the body of the inference POST request. Once this is included, my output is exactly how I hoped: exactly the JSON and only the JSON.

{
  "question": "Name something you might find in a garage",
  "answers": [
    { "answer": "Tools", "score": 40 },
    { "answer": "Bike", "score": 20 },
    { "answer": "Work Bench", "score": 12 },
    { "answer": "Vacuum", "score": 8 },
    { "answer": "Lawn Mower", "score": 5 },
    { "answer": "Storage Containers", "score": 5 }
  ]
}

Note generation takes longer when JSON mode is enabled. This is a consequence of how that guided decoding works, which is worth a blog post in it’s own right.

We haven’t talked about the Featherless API we’ve been using for inference, but hold on to that thought - more on that in a bit.

Part 2 - Fuzzy Answer Matching

The other challenge is matching players answers to the list of official answers, as they don’t always match, in our running example from the video above, the question “Name something that might be wobbly” had the following answers

Where “A stool” was accepted in place of “Furniture” and “A baby” was accepted as a version of “Person / a drunk”.

It’s relatively easy to devise a prompt and post-processing that captures the gist of the idea. Here’s a short python program we used during testing to let you iterate through prompt and model choice by letting an answer you supply on the command line is judged.

But catching the comedic subtlety of the game is something else entirely. Despite many prompt iterations and many different models I tried (including Llama 3.1 405B!), I couldn’t get the judge to accept “A baby” as a variation of the answer “Person / A drunk”. If you have a different result, I’d love to hear it in the comments!

Choosing an inference provider

There is a growing list of inference providers to choose from. Every foundation model company provides API access to their models (OpenAI, Anthropic, etc.), companies focussed just on inference provision (friendli.ai, replicate.ai, together.ai), and aggregators (e.g. openrouter.ai).

What’s unique about featherless.ai is the number of models available serverlessly, i.e. without paying for dedicated GPUs. With the exception of featherless, all other serverless providers only make the most popular models are available, since, behind the scenes, there is dedicated infrastructure to those models, those costs are just being amortized across a large enough group of consumers. Featherless is different. Every fine-tune of a large class of models (i.e. all fine-tunes of a specific set of base architectures) is available serverlessly. Our goal is to make every public model on HuggingFace available for serverless inference and we currently have over 2k models available.

The next closest service in model count is openrouter.ai, which aggregates providers, and has less than 10% of the available models.

Model Selection and Testing

Play testing is an important part of any game development, and as we were playing with Featherless Feud, we tried a number of different models. The primary issues were a lack of diverse answers or answers that were too ambiguous for the game format. We’ve left the utility box that overrides the model in the lower-right hand corner of the game; this showcases some of the strength of the featherless platform: take any of our 2k models from our models page, drop it into that selector, and the game is instantly updated. When contrasted that just model download times are tens of minutes for small models, you might appreciate this feat.

We tested a range of model architectures, sizes, and data-sets, including

• failspy/Meta-Llama-3-8B-Instruct-abliterated-v3 - we chose this model to explore if the abilteration approach would increase the variety of questions in an interesting way (inconclusive)

• qwen/qwen1.5-32b - The Qwen models feel very different than the Llama models - maybe that’s the culture of Alibaba versus Meta coming through …

• meta-llama/Meta-Llama-3.1-70B-Instruct: While this 70B model offered superior judgment in evaluating user answers, its slower response time made it less suitable for rapid iteration during gameplay. However, it was useful in scenarios where answer accuracy was critical.

• alpindale/magnum-72b-v1: Initially, this 72B model seemed promising due to its creative output. However, its tendency for erratic responses made it unsuitable for the structured requirements of Featherless Feud.

• Kooten/Mistral-Nemo-Instruct-2407-norefuse-OAS

• Sao10K/L3-8B-Stheno-v3.2: This 8B model was selected for its optimal balance between speed and output quality. It consistently produced valid JSON responses, making it a reliable choice for generating game content efficiently.

• anthracite-org/magnum-32b-v2 - a qwen2-32b fine-tune intended for role-play

At a high-level, the more RP-focussed models tended to generate more variety of questions, and the bigger models didn’t generate content with a sufficient jump in quality to justify the longer running time. So we settled on Stheno as it provided a balance of variety of output with fastest inference time.

Try running the game with some of the models and let us know in the comments what you differences you notice!

Wrapping up

We learned a ton in this journey, and hope you did too. Perhaps most surprising is that capturing the comedic subtlety of Family Feud escaped us even when using the more capable (i.e. larger) open-source LLMs.

Thanks for reading! Let us know how you enjoyed this article, either in the comments, or with an email to hello@featherless.ai, and head on over to featherless.ai to experiment with the models listed here and more.

And again, check out the running game here

Since our initial launch in June, we’ve been adding architectures, with fanfare only in our discord. But with Mistral Nemo 12B and Qwen2 32B becoming supported this week, taking the # of supported architectures to 12 and the total inferencible model count* to nearly 2k (1,922 at time of writing), this post seems overdue.

The full list of supported architectures is available on our about page, but the timeline is this

When you break that model registration out over time, it looks something like this

That smaller models are easier (take less time and money) to fine-tune accounts for a larger proportion of 7B and 8Bs (versus 70B, 72B). Likewise the older the model is, the more time folks have had to fine tune (hence many Llama 2 tunes, but few Nemos and Qwens).

If you want to weigh in on what model architecture we’re supporting next, join our discord.

Also a plug for fine-tuners: we’re working on a set of features that will be of benefit to model creators. If you are doing some fine-tuning, we’d love to connect with you for feedback on these upcoming features.

Like being able to access over 450+ models from huggingface and know someone who would love to try?

Refer them to featherless.ai and both of you get the $10 OFF your next monthly bill!

Refer 12 of your friends and you can have a full year off our basic plan! (The discount stacks!)

Enjoy 🤗 your models with featherless.ai !

Reminder: All our models do not log any of your messages prompt or completion 😉

Earlier this year, we took the world by storm when we announced that our Eagle model had beaten Meta’s Llama-2 while taking less training time, being the world’s most efficient model.

While Eagle still packs a powerful punch, and has been helping diverse use-cases from multi-lingual, to content moderation, gaming, and role-play, we’ve been working on something new, to bring our insights on efficiency to a much broader realm.

Just this Friday, we launched Featherless AI, which enables serverless inference of every Llama-3 8B and 70B model on Hugging Face we grabbed our hands.

That’s over 475 models. With many more being added daily.

Allowing anyone to quickly experiment, try, and choose the latest and best models, from huggingface. Starting from $10 / month.

Previously, to use the even the smallest fine-tunes requires dedicated hardware, which translates to real hosting costs, whether you’re experimenting with a model or ramping up production use. This is a barrier to a host of use cases particularly agents where each step in the agent computation might benefit from a particular model.

The goal of featherless is to make every model on HuggingFace available serverless and with these Llama & RWKV based models, we’re a big step of the way there.

With featherless, you can experiment with an entirely new range of models at completely different economics.

• Check out the site: featherless.ai

• Retweet our launch tweet,

• Support our product hunt launch

There are more than 100,000 distinct language models on the hugging face hub.

This is the output of an enormous amount of creative energy: built by over 10k AI enthusiasts, these models include impressive attempts to improve upon the best known language models like ChatGPT.

A lot of airtime goes to innovations on technical elements of language models (e.g. context length). And while important, that a huge part of what left with a collection there are a great many of domain specific LLMs like

• for specific languages (e.g. for German, Russian or Chinese)

• for creative-writing (e.g tdrussell’s Llama-3-70b-Instruct-Storywriter)

• with detailed medical knowledge (e.g. BioMistral/BioMistral-7B)

• can understand SEC filings, (e.g. arcee-ai/Llama-3-SEC)

• legal (e.g. umarbutler/open-australian-legal-llm - a model trained on a dataset of Australian Law curated by the Australian Attorney General’s office!)

• novelty / character (e.g. MopeyMule)

but it’s hard to use

So how do you use these things?

Despite that HuggingFace is the defacto place to host models, you’re hard pressed to use them there. If you’ve spent time on the site, you may have forgotten that there is a specific part of the model card designed to let you test the model: it’s typically disabled for models 8B and up, which is the vast majority of models.

This kind of UX is a fact of that running these models requires operating expensive hardware (i.e. GPUs). You can rent these GPUs, but you’re looking at at least $2 / hour, and that would only cover you for the smaller models.

If you have a budget to experiment with, you can try launching the model on a dedicated service. However this will also require your patience; the most natural service to do this is HuggingFace’s inference endpoints service. Which I haven’t gotten it to work, despite the suggestion I should be able to launch a model in a few clicks.

Openrouter is probably the cloud service with the most options and offers per-token pricing. but it doesn’t have any of the models listed above. Nor does it let you bring your own model.

You can try and run it locally (and there are a host of tools that have significantly simplify the process - gpt4all, ollama and cortex are the more popular. But they still require technical orientation, patience, and, most importantly, powerful computing hardware.

Experiment Faster with Featherless

The goal of featherless is to make every LLM on hugging face available serverlessly. Right now, our collection is up to 1,501 models, making it the largest collection of models available for inference from any inference provider.

You can test any of the models on the site, but we’re expecting that you’ll plug this directly into a client, whether that’s to chat with it as a human e.g. via Typing Mind, Jan, or Silly Tavern, or you’ll use the API directly, e.g. in a raw python program, or in some higher-level framework like Lang Chain or Llama Index

Checkout our terms of service and privacy policy.