SHOW HN: FINE-TUNED 110M ENCODER BEAT 7B LLMS AND HYBRID SEARCH FOR NIST MAPPING

**Hacker Pranks Exclusive: Open-Source Framework Revolutionizes Cybersecurity Compliance Mapping**

In the ever-evolving world of cybersecurity, identifying vulnerabilities and mapping them to compliance controls is a daunting task for many organizations. However, a recent breakthrough by Applied Inference Lab's Compliance Control Embedding Stack (CCES) has shed new light on this challenge. This open-source machine learning framework, developed by James Pusateri and Middle Coast Software Inc., has successfully fine-tuned a 110M encoder to outperform 7B Large Language Models (LLMs) and hybrid search methods in NIST mapping.

**The Problem: Bridging the Gap between Vulnerability Descriptions and Compliance Controls**

Technical vulnerability descriptions, such as memory corruption and authentication bypasses, often inhabit a different vocabulary space than administrative compliance controls, like flaw remediation and boundary protection. This disparity creates a significant operational bottleneck in cybersecurity compliance programs. Correlating technical vulnerability scans with regulatory compliance frameworks is a complex and time-consuming task that requires a deep understanding of both technical and administrative concepts.

**Introducing Compliance Control Embedding Stack (CCES)**

CCES is an open-source machine learning framework designed to address this challenge. By leveraging a novel approach to mapping technical software vulnerabilities (CVE descriptions) to formal cybersecurity compliance controls (NIST SP 800-53 Rev 5 and CMMC), CCES has successfully bridged the gap between these two seemingly disparate domains. This repository provides the complete model generation, contrastive fine-tuning, evaluation, and GGUF quantization pipeline developed for AutoRMF.

**Evaluating CCES against NIST SP 800-53 Rev 5**

The evaluation process involved assessing the fine-tuned model against a 2,000-sample golden validation dataset. The results, which were conducted against the full NIST SP 800-53 Rev 5 catalog (1,196 controls), revealed critical insights that contradict common enterprise search assumptions. Specifically, the evaluations demonstrated that the fine-tuned 110M encoder outperformed 7B LLMs and hybrid search methods in NIST mapping.

**Getting Started with CCES**

To leverage the power of CCES, users must first create and activate a virtual environment using Python 3.10 or later. The repository includes a comprehensive guide to setting up the environment and executing the scripts. Once the environment is activated, users can initialize or verify the schema from catalog seed files using the following script:

```python python -m src init -c /path/to/catalog/seed/files ```

**Using CCES: A Step-by-Step Guide**

To evaluate the fine-tuned model against the 2,000-sample golden validation dataset, users can run the following script:

```python python -m src evaluate -d data/golden_validation_set.json ```

To reproduce the reranking experiment using a local Cross-Encoder, users can run the following script:

```python python -m src rerank -m /path/to/local_cross_encoder ```

**Conclusion**

The Compliance Control Embedding Stack (CCES) is a groundbreaking open-source framework that has revolutionized the way we map technical software vulnerabilities to formal cybersecurity compliance controls. By fine-tuning a 110M encoder, CCES has outperformed 7B Large Language Models (LLMs) and hybrid search methods in NIST mapping. With its comprehensive guide and user-friendly interface, CCES is an invaluable resource for organizations seeking to streamline their cybersecurity compliance programs. Whether you're a seasoned security professional or a newcomer to the field, CCES is an essential tool for any organization looking to stay ahead of the curve in the ever-evolving world of cybersecurity.

**License and Copyright**

This project is licensed under the Apache License, Version 2.0. See the LICENSE file for details. Copyright 2026 Middle Coast Software Inc.

**Getting Involved**

The CCES repository is open to contributions from the cybersecurity and machine learning communities. If you're interested in contributing to the project or have any questions, please don't hesitate to reach out to the authors or join the discussion on the CCES GitHub page.