Filter by type:
Filter by tags:
allenai_olmo-3.1-32b-think
The **Olmo-3.1-32B-Think** model is a large language model (LLM) optimized for efficient inference using quantized versions. It is a quantized version of the original **allenai/Olmo-3.1-32B-Think** model, developed by **bartowski** using the **imatrix** quantization method.
### Key Features:
- **Base Model**: `allenai/Olmo-3.1-32B-Think` (unquantized version).
- **Quantized Versions**: Available in multiple formats (e.g., `Q6_K_L`, `Q4_1`, `bf16`) with varying precision (e.g., Q8_0, Q6_K_L, Q5_K_M). These are derived from the original model using the **imatrix calibration dataset**.
- **Performance**: Optimized for low-memory usage and efficient inference on GPUs/CPUs. Recommended quantization types include `Q6_K_L` (near-perfect quality) or `Q4_K_M` (default, balanced performance).
- **Downloads**: Available via Hugging Face CLI. Split into multiple files if needed for large models.
- **License**: Apache-2.0.
### Recommended Quantization:
- Use `Q6_K_L` for highest quality (near-perfect performance).
- Use `Q4_K_M` for balanced performance and size.
- Avoid lower-quality options (e.g., `Q3_K_S`) unless specific hardware constraints apply.
This model is ideal for deploying on GPUs/CPUs with limited memory, leveraging efficient quantization for practical use cases.
Repository: localai
insightface-buffalo-s
Small insightface pack (SCRFD-500MF detector + MBF 512-d embedder +
genderage, ~159MB). Good fit for mid-range CPU deployments.
NON-COMMERCIAL RESEARCH USE ONLY.
Repository: localaiLicense: insightface-non-commercial
insightface-opencv-int8
Int8-quantized OpenCV Zoo face pair (YuNet int8 + SFace int8, ~12MB).
Roughly 3x smaller and noticeably faster on CPU than the fp32 variant
at comparable accuracy for face tasks. APACHE 2.0 — commercial-safe.
Weights are downloaded on install via LocalAI's gallery mechanism.
Repository: localaiLicense: apache-2.0
wespeaker-resnet34
Speaker recognition with WeSpeaker's ResNet34 trained on VoxCeleb,
exported to ONNX. 256-d embeddings, CPU-friendly — avoids the
PyTorch runtime entirely (onnxruntime only). APACHE 2.0.
Pair with the `speaker-recognition` backend's OnnxDirectEngine.
Use when ECAPA-TDNN's torch dependency is undesirable (small
images, edge deployments).
Repository: localaiLicense: apache-2.0
qwen3-30b-a1.5b-high-speed
This repo contains the full precision source code, in "safe tensors" format to generate GGUFs, GPTQ, EXL2, AWQ, HQQ and other formats. The source code can also be used directly.
This is a simple "finetune" of the Qwen's "Qwen 30B-A3B" (MOE) model, setting the experts in use from 8 to 4 (out of 128 experts).
This method close to doubles the speed of the model and uses 1.5B (of 30B) parameters instead of 3B (of 30B) parameters. Depending on the application you may want to use the regular model ("30B-A3B"), and use this model for simpler use case(s) although I did not notice any loss of function during routine (but not extensive) testing.
Example generation (Q4KS, CPU) at the bottom of this page using 4 experts / this model.
More complex use cases may benefit from using the normal version.
For reference:
Cpu only operation Q4KS (windows 11) jumps from 12 t/s to 23 t/s.
GPU performance IQ3S jumps from 75 t/s to over 125 t/s. (low to mid level card)
Context size: 32K + 8K for output (40k total)
Repository: localaiLicense: apache-2.0
qwen3-55b-a3b-total-recall-deep-40x
WARNING: MADNESS - UN HINGED and... NSFW. Vivid prose. INTENSE. Visceral Details. Violence. HORROR. GORE. Swearing. UNCENSORED... humor, romance, fun.
Qwen3-55B-A3B-TOTAL-RECALL-Deep-40X-GGUF
A highly experimental model ("tamer" versions below) based on Qwen3-30B-A3B (MOE, 128 experts, 8 activated), with Brainstorm 40X (by DavidAU - details at bottom of this page).
These modifications blow the model (V1) out to 87 layers, 1046 tensors and 55B parameters.
Note that some versions are smaller than this, with fewer layers/tensors and smaller parameter counts.
The adapter extensively alters performance, reasoning and output generation.
Exceptional changes in creative, prose and general performance.
Regens of the same prompt - even with the same settings - will be very different.
THREE example generations below - creative (generated with Q3_K_M, V1 model).
ONE example generation (#4) - non creative (generated with Q3_K_M, V1 model).
You can run this model on CPU and/or GPU due to unique model construction, size of experts and total activated experts at 3B parameters (8 experts), which translates into roughly almost 6B parameters in this version.
Two quants uploaded for testing: Q3_K_M, Q4_K_M
V3, V4 and V5 are also available in these two quants.
V2 and V6 in Q3_k_m only; as are: V 1.3, 1.4, 1.5, 1.7 and V7 (newest)
NOTE: V2 and up are from source model 2, V1 and 1.3,1.4,1.5,1.7 are from source model 1.
Repository: localaiLicense: apache-2.0
qwen3-22b-a3b-the-harley-quinn
WARNING: MADNESS - UN HINGED and... NSFW. Vivid prose. INTENSE. Visceral Details. Violence. HORROR. GORE. Swearing. UNCENSORED... humor, romance, fun.
Qwen3-22B-A3B-The-Harley-Quinn
This repo contains the full precision source code, in "safe tensors" format to generate GGUFs, GPTQ, EXL2, AWQ, HQQ and other formats. The source code can also be used directly.
ABOUT:
A stranger, yet radically different version of Kalmaze's "Qwen/Qwen3-16B-A3B" with the experts pruned to 64 (from 128, the Qwen 3 30B-A3B version) and then I added 19 layers expanding (Brainstorm 20x by DavidAU info at bottom of this page) the model to 22B total parameters.
The goal: slightly alter the model, to address some odd creative thinking and output choices.
Then... Harley Quinn showed up, and then it was a party!
A wild, out of control (sometimes) but never boring party.
Please note that the modifications affect the entire model operation; roughly I adjusted the model to think a little "deeper" and "ponder" a bit - but this is a very rough description.
That being said, reasoning and output generation will be altered regardless of your use case(s).
These modifications pushes Qwen's model to the absolute limit for creative use cases.
Detail, vividiness, and creativity all get a boost.
Prose (all) will also be very different from "default" Qwen3.
Likewise, regen(s) of the same prompt - even at the same settings - will create very different version(s) too.
The Brainstrom 20x has also lightly de-censored the model under some conditions.
However, this model can be prone to bouts of madness.
It will not always behave, and it will sometimes go -wildly- off script.
See 4 examples below.
Model retains full reasoning, and output generation of a Qwen3 MOE ; but has not been tested for "non-creative" use cases.
Model is set with Qwen's default config:
40 k context
8 of 64 experts activated.
Chatml OR Jinja Template (embedded)
Four example generations below.
IMPORTANT:
See usage guide / repo below to get the most out of this model, as settings are very specific.
If not set correctly, this model will not work the way it should.
Critical settings:
Chatml or Jinja Template (embedded, but updated version at repo below)
Rep pen of 1.01 or 1.02 ; higher (1.04, 1.05) will result in "Harley Mode".
Temp range of .6 to 1.2. ; higher you may need to prompt the model to "output" after thinking.
Experts set at 8-10 ; higher will result in "odder" output BUT it might be better.
That being said, "Harley Quinn" may make her presence known at any moment.
USAGE GUIDE:
Please refer to this model card for
Specific usage, suggested settings, changing ACTIVE EXPERTS, templates, settings and the like:
How to maximize this model in "uncensored" form, with specific notes on "abliterated" models.
Rep pen / temp settings specific to getting the model to perform strongly.
https://huggingface.co/DavidAU/Qwen3-18B-A3B-Stranger-Thoughts-Abliterated-Uncensored-GGUF
GGUF / QUANTS / SPECIAL SHOUTOUT:
Special thanks to team Mradermacher for making the quants!
https://huggingface.co/mradermacher/Qwen3-22B-A3B-The-Harley-Quinn-GGUF
KNOWN ISSUES:
Model may "mis-capitalize" word(s) - lowercase, where uppercase should be - from time to time.
Model may add extra space from time to time before a word.
Incorrect template and/or settings will result in a drop in performance / poor performance.
Can rant at the end / repeat. Most of the time it will stop on its own.
Looking for the Abliterated / Uncensored version?
https://huggingface.co/DavidAU/Qwen3-23B-A3B-The-Harley-Quinn-PUDDIN-Abliterated-Uncensored
In some cases this "abliterated/uncensored" version may work better than this version.
EXAMPLES
Standard system prompt, rep pen 1.01-1.02, topk 100, topp .95, minp .05, rep pen range 64.
Tested in LMStudio, quant Q4KS, GPU (CPU output will differ slightly).
As this is the mid range quant, expected better results from higher quants and/or with more experts activated to be better.
NOTE: Some formatting lost on copy/paste.
WARNING: NSFW. Vivid prose. INTENSE. Visceral Details. Violence. HORROR. GORE. Swearing. UNCENSORED... humor, romance, fun.
Repository: localaiLicense: apache-2.0
menlo_lucy
Lucy is a compact but capable 1.7B model focused on agentic web search and lightweight browsing. Built on Qwen3-1.7B, Lucy inherits deep research capabilities from larger models while being optimized to run efficiently on mobile devices, even with CPU-only configurations.
We achieved this through machine-generated task vectors that optimize thinking processes, smooth reward functions across multiple categories, and pure reinforcement learning without any supervised fine-tuning.
Repository: localaiLicense: apache-2.0
menlo_lucy-128k
Lucy is a compact but capable 1.7B model focused on agentic web search and lightweight browsing. Built on Qwen3-1.7B, Lucy inherits deep research capabilities from larger models while being optimized to run efficiently on mobile devices, even with CPU-only configurations.
We achieved this through machine-generated task vectors that optimize thinking processes, smooth reward functions across multiple categories, and pure reinforcement learning without any supervised fine-tuning.
Repository: localaiLicense: apache-2.0
wan-2.1-t2v-1.3b-ggml
Wan 2.1 T2V 1.3B — text-to-video diffusion model, GGUF-quantized for the
stable-diffusion.cpp backend. Generates short (33-frame) 832x480 clips
from a text prompt. Cheapest Wan variant, suitable for CPU-offloaded
inference with ~10 GB of usable RAM.
Repository: localaiLicense: apache-2.0
wan-2.1-i2v-14b-480p-ggml
Wan 2.1 I2V 14B 480P — image-to-video diffusion, GGUF Q4 quantization.
Animates a reference image into a 33-frame 480p clip. Requires more
RAM than the 1.3B T2V variant; CPU offload enabled by default.
Repository: localaiLicense: apache-2.0
aevum-0.6b-finetuned
**Model Name:** Aevum-0.6B-Finetuned
**Base Model:** Qwen3-0.6B
**Architecture:** Decoder-only Transformer
**Parameters:** 0.6 Billion
**Task:** Code Generation, Instruction Following
**Languages:** English, Python (optimized for code)
**License:** Apache 2.0
**Overview:**
Aevum-0.6B-Finetuned is a highly efficient, small-scale language model fine-tuned for code generation and task following. Built on the Qwen3-0.6B foundation, it delivers strong performance—achieving a **HumanEval Pass@1 score of 21.34%**—making it the most parameter-efficient sub-1B model in its category.
**Key Features:**
- Optimized for low-latency inference on CPU and edge devices.
- Fine-tuned on MBPP and DeepMind Code Contests for superior code generation accuracy.
- Ideal for lightweight development, education, and prototyping.
**Use Case:**
Perfect for developers and researchers needing a fast, compact, and open model for Python code generation without requiring high-end hardware.
**Performance Benchmark:**
Outperforms larger models in efficiency: comparable to models 10x its size in task accuracy.
**Cite:**
@misc{aveum06B2025, title={aevum-0.6B-Finetuned: Lightweight Python Code Generation Model}, author={anonymous}, year={2025}}
**Try it:**
Use via Hugging Face `transformers` library with minimal setup.
👉 [Model Page on Hugging Face](https://huggingface.co/Aevum-Official/aveum-0.6B-Finetuned)
Repository: localaiLicense: apache-2.0