Back to Blog
    detection engineeringeducation

    How We Build and Test YARA Rules at Scale

    ·16 min read

    How We Build and Test YARA Rules at Scale

    Most organizations treat YARA as a research tool. Something a malware analyst runs manually against a suspect file during an investigation. A few hundred rules someone downloaded from GitHub and dropped into a directory three years ago.

    We run over 6,000 rules in production, scanning every file that hits a monitored directory on every endpoint. New rules get generated from live malware samples using AI, validated through a multi stage consensus pipeline compiled against YARA-X, and deployed to agents automatically. When a file matches, the alert shows up in our SIEM within seconds.

    This isn't a theoretical workflow. It runs today across Windows, Linux, and macOS endpoints. Here's how it works.

    The Rule Library

    Our YARA library pulls from four sources. Each one covers a different part of the threat landscape, and the overlap between them is intentional. Defense in depth applies to signatures too.

    SourceRulesCoverage
    YARA Forge Core4,925Aggregated from 40+ open source repositories
    Nextron signature-base602APT campaigns, hacking tools, webshells, exploits
    PookaSec Goibniu332Custom rules from live malware analysis
    ReversingLabs293Ransomware families (LockBit, BlackCat, WannaCry, 245 more)

    The YARA Forge Core set provides the broadest coverage by aggregating community rules from dozens of researchers. We owe a particular debt to Florian Roth and the team at Nextron Systems. Florian built YARA Forge, created the Detection Rule License that makes sharing detection content commercially viable, and maintains signature-base, one of the most comprehensive public YARA collections in existence. The Nextron rules are the surgical layer in our library, covering named threat actors (APT28, APT29, Lazarus, Turla, Winnti) and the tools those actors use (Mimikatz, Cobalt Strike, PsExec variants). ReversingLabs gives us deep ransomware coverage, which matters because ransomware is the threat most of our clients actually face. And the Goibniu rules cover the long tail: fresh samples from MalwareBazaar that the curated sets haven't caught up to yet.

    Goibniu currently covers 25 malware families, with the heaviest coverage on RATs (njRAT, RemcosRAT, AsyncRAT, QuasarRAT, DCRat, NanoCore), stealers (AgentTesla, RedLine, Vidar, Stealc, SalatStealer), and commodity loaders (Emotet, Formbook, QakBot). We also carry rules for targeted threats like Lazarus, ValleyRAT, and LockBit.

    Licensing

    Aggregating rules from 40+ sources means aggregating 40+ licenses. We run a license auditor against every rule in the library to classify each one by commercial compatibility before anything reaches production.

    The breakdown across our 4,925 YARA Forge Core rules:

    LicenseRulesCommercial Use
    Detection Rule License 1.11,694Yes, with attribution
    CC-BY-SA 4.0 (Malpedia)1,557Yes, with ShareAlike
    BSD 2 Clause857Yes, with copyright notice
    Apache 2.0175Yes, with notice
    GPLv3178Yes, copyleft applies to derivatives
    MIT16Yes
    BSD 3 Clause29Yes
    No explicit license419Unknown

    Zero CC-BY-NC rules. That was the primary concern going in. The YARA Forge Core bundle aggregates from repositories that all permit commercial use. Florian Roth's curation excludes sources with restrictive licenses (Elastic License v2, for example) before the bundle is built.

    The 419 rules with no explicit license come from researchers who published rules on GitHub without a LICENSE file. Under copyright law, no license means all rights reserved. In strict mode, our auditor strips these rules from the bundle before deployment, leaving 4,506 rules with clear commercial terms. In default mode, they stay in because the practical risk of a YARA rule author suing a security vendor for detecting malware with their rule is negligible, but we track them separately and can filter at any time.

    The DRL 1.1 rules require attribution (preserving the author field in rule metadata), which we do. The CC-BY-SA rules from Malpedia require ShareAlike on derivatives. Since we use those rules as is for scanning rather than creating derivative works from them, this is satisfied. The GPLv3 rules from CAPEv2 carry copyleft obligations on distribution of modified versions, but internal use for scanning is not distribution.

    Our own Goibniu rules, Nextron signature-base (DRL 1.1), and ReversingLabs (MIT) are all clear for commercial use. The license auditor runs as part of the pipeline and produces a JSON report for every deployment, so the compliance state is always auditable.

    Modules and Compilation

    We compile all rules against YARA-X, VirusTotal's Rust rewrite of the YARA engine. Faster compilation, precise error messages, and broader module support than the original engine.

    We allow the dotnet and dex modules. The dotnet module is particularly valuable because a large percentage of the RAT families we track (AsyncRAT, QuasarRAT, DCRat, njRAT) are .NET binaries. Rules that check dotnet.is_dotnet or inspect .NET assembly metadata are more precise and generate fewer false positives than pure string matching against the same samples. The dex module covers Android malware, which is a growing part of the sample feeds. We block cuckoo and magic modules since they require runtime dependencies that don't exist on endpoints.

    The dotnet module has a known blind spot: .NET obfuscators and protectors (ConfuserEx, .NET Reactor, Babel, Eazfuscator) can mangle CLR metadata enough that dotnet.is_dotnet returns false or assembly name checks fail on a binary that is still functionally .NET. We handle this by never writing rules that rely solely on the module. Every Goibniu rule that uses dotnet conditions also carries string based detection as a parallel path. The condition structure is typically (dotnet.is_dotnet and 4 of ($dotnet_*)) or (6 of ($s*)), so an obfuscated sample that breaks metadata parsing still gets caught by the string layer. The curated rule sets from YARA Forge and Nextron provide additional coverage for the same families using pure string and hex patterns, which means an obfuscated variant has to defeat multiple independent rule sets to avoid detection entirely.

    The syntax validator runs a four level fallback chain when compiling rules:

    1. YARA-X Python library (yara_x.compile()) as the primary compiler
    2. YARA-X CLI (yr check) as first fallback
    3. yara-python (yara.compile()) as legacy fallback
    4. YARA CLI (yara -C) as final fallback

    Every rule in the library compiles against YARA-X. If the primary compiler rejects a rule, the fallback chain catches engine specific quirks, but the YARA-X result drives the validation score. Rules that fail YARA-X compilation go to manual review regardless of whether they pass a fallback compiler.

    We also run yr fmt --check as a non blocking formatting quality check. It doesn't affect the validation score, but it flags rules with inconsistent indentation or style issues so they can be cleaned up during review.

    How Rules Get Written

    Hand authored rules follow a specific template. Every rule has structured metadata, categorized strings, and a condition that puts cheap checks first.

    rule MAL_Ransomware_PSRansom_Script {
        meta:
            description = "Detects PSRansom ransomware simulation PowerShell script"
            author = "PookaSec"
            date = "2026-02-08"
            reference = "https://github.com/JoelGMSec/PSRansom"
            mitre_attack = "T1486"
            score = 75
        strings:
            $x1 = "Invoke-AESEncryption" ascii wide
            $x2 = "PSRansom" ascii wide nocase
            $s1 = "Encrypt" ascii wide
            $s2 = ".onion" ascii wide
            $s3 = "YOUR FILES HAVE BEEN ENCRYPTED" ascii wide nocase
        condition:
            any of ($x*) or 3 of ($s*)
    }
    

    The string prefixes aren't cosmetic. $x* strings are highly specific: if any one of them matches, the file is almost certainly the thing we're looking for. $s* strings are weaker individually but strong in combination. A file containing "Encrypt" isn't suspicious. A file containing "Encrypt" and ".onion" and "YOUR FILES HAVE BEEN ENCRYPTED" is a ransom note.

    We enforce a minimum 4 byte atom length on every string. Short strings like "MZ" or \x00\x00 generate massive numbers of partial matches and destroy scan performance. If a string can't provide 4 bytes of unique content, it gets rewritten or dropped.

    Condition ordering matters. filesize < 2MB is essentially free to check and eliminates most of the filesystem before any string scanning happens. Module free alternatives like uint16(0) == 0x5A4D (checking for an MZ header at offset zero) let us filter by file type without the PE module dependency. For .NET samples, dotnet.is_dotnet is now available and more reliable than the magic bytes check. String matching happens last, only on files that pass the cheaper filters.

    Goibniu: The Autogen Pipeline

    Hand writing rules works for known threats with published analysis. It doesn't scale to the volume of new samples that appear daily on MalwareBazaar.

    We call our autogen pipeline Goibniu, after the divine smith of the Tuatha De Danann in Irish mythology. Our pipeline forges detection signatures from real malware samples. It runs as a six phase bash orchestrator that coordinates Python scripts for each stage:

    MalwareBazaar API (900 samples queried across 9 tags)
        |
        v
    Sample Fetcher (filtered by tag, file type, size, VT detections)
        |
        v
    Deduplicator (4 layers: SHA256, ssdeep, TLSH, string overlap)
        |
        v
    Sample Analyzer (static analysis, native or Docker isolated)
        |
        v
    Rule Generator (LLM with template fallback)
        |
        v
    Validator Consensus (syntax via YARA-X + FP risk + coverage)
        |
        v
    Deploy to Wazuh agents (auto deploy or manual review queue)
    

    Phase 1: Acquisition. The fetcher queries MalwareBazaar across nine tags: ransomware, banker, APT, trojan, RAT, loader, stealer, infostealer, backdoor. Each tag returns up to 100 recent samples. Only executables (PE, ELF, Mach-O) under 10 MB pass the filter. Samples need at least 5 VirusTotal detections to cut the noise of unconfirmed uploads. A typical run queries 900 samples and filters down to 20-30 candidates.

    Phase 2: Deduplication. Four layers prevent wasted work. SHA256 catches exact duplicates against all existing rules in the library (currently indexed against 6,757 hashes). ssdeep catches repacks where the binary is slightly modified but structurally identical. TLSH catches deeper variants with the same code but different packing. String overlap analysis catches samples that would produce rules covering the same detection surface as an existing rule. In our most recent batch, dedup eliminated 19 of 30 downloaded samples (63%), leaving 11 genuinely new samples for analysis.

    Phase 3: Analysis. Static analysis extracts strings, identifies embedded artifacts, maps imports, calculates entropy, and checks packing indicators. The analyzer runs natively or in a Docker container with no network access. Output includes categorized strings (URLs, C2 indicators, registry keys, mutexes, crypto wallet addresses) and a ranked list of YARA candidate strings, the most distinctive strings that differentiate this sample from legitimate software.

    Phase 4: Rule Generation. The analysis profile goes to an LLM, which generates a YARA rule targeting the sample's most distinctive characteristics. The prompt specifies naming conventions, metadata requirements, string selection criteria (5-15 distinctive strings, avoid generic Windows API names), and module permissions (dotnet and dex allowed, cuckoo and magic forbidden). If AI generation fails or is disabled, a template engine produces a simpler rule from the candidate strings directly. In our latest batch, all 11 rules were generated via the LLM.

    Phase 5: Validation. Three independent validators vote on every rule.

    Phase 6: Deployment. Rules that pass auto deploy get copied to the shared YARA rules directory and synced to all Wazuh agents. Rules in the review queue get a human look before deployment.

    Validation Before Deployment

    No rule reaches an endpoint without passing three independent validators, each checking a different failure mode.

    Syntax validator (40% weight): Does the rule compile under YARA-X? Does it use any forbidden modules? Are all strings well formed? Is the rule properly structured with meta, strings, and condition sections? A formatting check via yr fmt --check flags style issues but doesn't affect the score. This catches the obvious breakage before anything else runs.

    False positive risk validator (30% weight): Are the strings specific enough? The validator scores string quality (penalizing generic/common strings), analyzes condition complexity (requiring filesize constraints and minimum string references), and checks overall FP risk. A score below 0.5 flags the rule as medium risk. Filesize constraints above 10 MB get flagged as too broad. Any pattern that looks like it could match normal system files gets penalized.

    Coverage validator (30% weight): Does this rule duplicate existing coverage? The validator indexes all existing rules (currently 10,727) and checks string overlap. If more than 50% of the new rule's strings already appear in an existing rule, it's flagged as redundant. It also tracks per family coverage to identify whether we already have adequate detection for that malware family. We want each rule to detect something no other rule catches. The 50% threshold is a blunt instrument. A new variant of a known family might legitimately share 60% of its strings with an existing rule yet carry one unique C2 domain or mutex that makes it worth tracking separately. Rules that land above the overlap threshold go to manual review rather than outright rejection for exactly this reason. As families evolve and string sets drift, this heuristic will need to get smarter, likely moving toward a weighted distance metric that values distinctive strings more heavily than common ones.

    Each validator returns a vote and a confidence score. The weighted average determines the outcome:

    Consensus ScoreAction
    0.8 or higherAuto deploy to production
    0.3 to 0.8Queued for manual review
    Below 0.3Rejected

    In our most recent batch of 11 new samples, 7 rules scored above 0.8 and auto deployed. The highest score was 0.979 (a QuasarRAT variant with unique strings and no overlap). Four rules went to manual review: two RemcosRAT variants with high string overlap against existing Remcos rules, one njRAT variant with similar overlap, and one AgentTesla rule that used the dotnet module in its condition but forgot the import "dotnet" statement at the top. YARA-X caught that last one cleanly with the error unknown identifier 'dotnet' ... the import "dotnet" statement is missing.

    Testing Against Real Malware

    Generating rules from malware samples is half the problem. The other half is confirming they actually detect those samples when deployed.

    Every Goibniu batch goes through a detection test. The test deploys the new rules to a staging agent, drops the original samples into monitored directories, and checks whether the Wazuh FIM module triggers a YARA match alert. If the rule was generated from sample X and doesn't detect sample X when scanned, something is wrong with the rule, not the sample.

    We also test against clean file sets to check for false positives. System binaries, common applications, document files. Any rule that fires on a clean file goes back through the generator with the false positive pattern documented as an exclusion.

    The detection flow in production:

    File appears in monitored directory
        |
        v
    Wazuh FIM detects file creation or modification
        |
        v
    YARA scanner runs all loaded rules against the file
        |
        v
    Match found: Wazuh alert (rule 861001, level 12)
        |
        v
    Filebeat ships alert to Graylog
        |
        v
    SOC analyst sees YARA match with rule name, file path, and host
    

    The scanning happens on the endpoint, not in the SIEM. The agent has the rules and the YARA binary locally. This means a file gets scanned the moment it appears, regardless of network connectivity or SIEM availability.

    What Breaks

    YARA at scale has failure modes you don't hit with a handful of rules.

    Performance degradation from bad atoms. One rule with a 2 byte hex pattern can slow down an entire scan by orders of magnitude. The YARA engine uses Aho-Corasick for string matching, and short patterns generate exponentially more candidate matches. We caught this early and enforce the 4 byte minimum as a hard gate in the syntax validator.

    AI generated rules with module errors. When AI generates rules that reference modules in the condition without the corresponding import statement, the rule looks syntactically plausible but fails to compile. YARA-X gives precise diagnostics for this: it names the unknown identifier, identifies the missing import, and suggests the fix. Two rules in our latest batch had this exact issue, and YARA-X caught both cleanly.

    Shell quoting and JSON escaping. YARA rules contain backslashes, quotes, and special characters. Embedding rule text in shell variables for validation scoring broke when those characters hit Python string interpolation. We fixed this by writing validator outputs to temp files and reading them back with json.load() instead of passing them through shell variable expansion. This is the kind of bug that only appears when you run the pipeline against real malware samples with complex strings, not when you test with simple rules.

    Rule conflicts from multiple sources. When you aggregate rules from four sources, you get rules that detect the same thing with different names, different conditions, and different false positive profiles. The coverage validator handles this for Goibniu rules, but the curated sources required a manual deduplication pass. We track overlap metrics to keep this manageable.

    Null family signatures. MalwareBazaar samples sometimes have a null signature field instead of an empty string or "unknown". Standard Python dict.get("signature", "unknown") returns None for this case because the key exists with a null value, which then crashes any code that calls .lower() on the result. The fix is dict.get("signature") or "unknown", using Python's falsy-or pattern instead of relying on the default parameter.

    Deployment

    Rules reach endpoints through Wazuh's shared configuration mechanism. The deploy script copies rules to the Wazuh manager's shared directory, and agents pull the updated ruleset on their next check in (within 10 minutes, or immediately if we force a sync).

    Local ruleset (1,228 files across 4 sources)
        |
        v
    SCP to SIEM server
        |
        v
    Docker copy into Wazuh manager container
        |
        v
    /var/ossec/etc/shared/default/yara/
        |
        v
    Agent sync (automatic, within 10 minutes)
        |
        v
    FIM scans use updated rules
    

    The deploy script verifies file counts match between local and remote after upload. In the latest deployment: 602 Loki rules, 293 ReversingLabs rules, 1 YARA Forge Core bundle (4,925 rules inside), and 332 PookaSec Goibniu rules.

    On Windows, the YARA binary lives at C:\Program Files\yara\yara64.exe and rules sync to the agent's shared directory. On Linux, YARA is at /usr/bin/yara with rules in /var/ossec/ruleset/yara/. The agent configuration specifies which directories trigger scans and where to find the rules. No manual intervention on endpoints after initial setup.

    What the Numbers Look Like

    Current pipeline metrics:

    MetricValue
    Total rules in production6,152
    YARA Forge Core4,925
    Nextron signature-base602
    PookaSec Goibniu332
    ReversingLabs293
    Goibniu malware families25
    Dedup index size6,757 hashes
    Coverage index size10,727 rules
    Auto deploy threshold0.8 consensus
    CompilerYARA-X 1.13.0
    Rule sources4
    Latest batch: generated11 rules
    Latest batch: auto deployed7 (64%)
    Latest batch: review queue4 (36%)
    Latest batch: rejected0

    The numbers that matter most aren't the rule counts. They're the detection rate (are rules catching what they should?) and the false positive rate (are they firing on things they shouldn't?). A library of 6,000 rules that generates 500 false positives a day is worse than 100 rules with zero false positives. The validator pipeline exists specifically to keep that ratio healthy.

    We track both. On the detection side, every Goibniu rule is tested against the sample it was generated from before deployment. Rules that don't detect their source sample never ship. Across the full library, we validate detection coverage against known malware samples during each batch run, and the curated sets (YARA Forge, Nextron, ReversingLabs) are tested upstream by their respective maintainers. On the false positive side, our canary file monitoring across endpoints has produced zero YARA false positives on clean system binaries and common applications since the current rule set was deployed. The validator pipeline's FP risk scoring, combined with the 4 byte atom minimum and filesize constraints, keeps that number where it is. When a rule does fire, it is almost always worth investigating.

    What's Next

    Goibniu is expanding to cover more sample sources beyond MalwareBazaar. Internal honeypot captures from our Cowrie deployment, samples from purple team exercises, and VirusTotal hunting rules are all candidates for automated rule generation.

    The point of all this isn't to have the most YARA rules. It's to have rules that detect real threats, validated against real malware, compiled against the latest engine, deployed automatically, and maintained continuously. That's the difference between a YARA library and a YARA capability.