minmodmon: A quickstart to local RWKV

In April we launched our RWKV-based model, EagleX v2. EagleX goes toe-to-toe with modern transformers on performance, while being much cheaper to run, and with an infinite context limit. The most common question I have personally seen about EagleX since then however has been, "How do I run it?".

Minmodmon is a small self-contained tool that lets you easily and quickly run RWKV-based models on Windows, on your GPU, locally. No dependencies required! Just download the latest release ZIP and run it!

AI for everyone

Our number 1 goal at Recursal is to make sure *everyone* receives the benefits of AI. We don't want you to have to be a technology expert to start running EagleX. There are already plenty of ways to run RWKV, but these tend to be aimed at more expert users.

Minmodmon was made to be used by anyone wanting to try out AI models, regardless of computer skill. It runs on Windows, requires no separate installs (no python, pip, etc), and needs no command-line expertise.

However, by design minmodmon is very limited. For a more feature-complete setup, the library web-rwkv that is used by minmodmon is also used in the excellent project ai00_server.

What you can do with it

Minmodmon is for use with other applications. You can't talk to a model directly through its web interface, but it integrates with common standards.

In particular, I recommend trying out minmodmon with SillyTavern. SillyTavern is an amazing AI chat application that lets you load in AI personas to talk with locally, completely free and open source.

This is still an early release. If you encounter any issues head over to our issue tracker and report them to us!

What's next

This release is just one step in our plans. We want both local and remote AI to be a seamless experience, putting you in control of what you use and where your data goes. The user experience still leaves much to be desired, but we have big plans in the works.

If you do not have a powerful GPU necessary to run local models, or just want better performance and larger models, we also recently launched a privacy-focused remote AI service, Featherless.