Xorcery

Xorcery is a Go library for scanning files and URLs against threat intelligence catalogs. It provides a simple, efficient API for detecting malicious files and URLs by comparing against MISP (Malware Information Sharing Platform) file catalogs and URLHaus URL catalogs.

Features

• File Scanning: Scan files by hash (MD5, SHA1, SHA256, CRC32) or by filename
• Byte Scanning: Scan raw byte data against threat catalogs
• URL Scanning: Comprehensive URL matching that catches malicious domains even with different paths or schemes
• Comprehensive URL Matching: Matches base/root URLs, handles www prefixes, and normalizes URLs
• Configurable: Use default catalog paths or specify custom paths
• Simple API: Clean, idiomatic Go API with both struct-based and convenience functions

Installation

go get github.com/PreciousNyasulu/xorcery

Quick Start

Basic Usage

package main

import (
    "fmt"
    "log"
    
    "github.com/PreciousNyasulu/xorcery/scanner"
)

func main() {
    // Create a scanner with default catalog paths
    s := scanner.NewScanner()
    
    // Scan a file by hash
    result, err := s.ScanFileByHash("abc123def456...", "sha256")
    if err != nil {
        log.Fatal(err)
    }
    
    if result.Type != "" {
        fmt.Printf("Threat detected: %s\n", result.Target)
    }
    
    // Scan a URL
    urlResult, err := s.ScanURL("https://example.com")
    if err != nil {
        log.Fatal(err)
    }
    
    if urlResult.Type != "" {
        fmt.Println("Malicious URL detected!")
    }
}

Using Convenience Functions

package main

import (
    "fmt"
    "log"
    
    "github.com/PreciousNyasulu/xorcery/scanner"
)

func main() {
    // Quick URL check
    isMalicious, err := scanner.CheckURL("https://suspicious-site.com")
    if err != nil {
        log.Fatal(err)
    }
    
    if isMalicious {
        fmt.Println("URL is in threat catalog")
    }
    
    // Scan file bytes
    fileData := []byte{0x00, 0x01, 0x02, ...}
    result, err := scanner.Bytes(fileData)
    if err != nil {
        log.Fatal(err)
    }
    
    if result.Type != "" {
        fmt.Println("Malicious file detected!")
    }
}

Catalog Setup

Xorcery requires threat intelligence catalogs in JSON format:

• File Catalog: MISP format JSON file (default: ./temp/file_catalog.json)
• URL Catalog: URLHaus format JSON file (default: ./temp/url_catalog.json)

You can use custom paths when creating a scanner:

s := scanner.NewScannerWithPaths(
    "/path/to/file_catalog.json",
    "/path/to/url_catalog.json",
)

URL Matching

Xorcery performs comprehensive URL matching that catches malicious domains even when only the root/base URL is provided:

• Exact Match: Normalized URLs match exactly
• Base URL Match: Scheme + hostname match (ignores path differences)
• Hostname Match: Hostnames match (ignores scheme differences like http vs https)
• www Prefix Handling: www.example.com matches example.com

Examples:

• "https://evil.com" matches "https://evil.com/path/to/malware"
• "http://evil.com" matches "https://evil.com"
• "https://www.evil.com" matches "https://evil.com"

Supported Hash Types

The library supports the following hash types for file scanning:

• md5
• sha1
• sha256
• crc32

Documentation

• Examples - More usage examples

Credits

This library uses threat intelligence data from:

• abuse.ch - URLHaus URL threat intelligence catalog
• MISP (Malware Information Sharing Platform) - File hash threat intelligence catalog

Special thanks to abuse.ch for providing the URLHaus threat intelligence feed, which is used for URL scanning functionality.

For more information, see ACKNOWLEDGMENTS.md.

License

See LICENSE file for details.