hexdev/hydrogen
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

first commit

Browse Source
This commit is contained in:
hexdev
2026-07-11 23:54:36 +07:00
commit b07562921b
40 changed files with 3813 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

520
HYDROGEN_SECURITY_MODULES.md Normal file
View File

@@ -0,0 +1,520 @@
# Hydrogen Custom Security Modules
This document explains how to add your own security checking modules to Hydrogen. It is based on the actual module loading flow implemented in `core/module_loader.py`, `core/base.py`, `core/schemas/modules.py`, and the example SSH module under `modules/ssh`.
## How Hydrogen Loads Modules
Hydrogen starts module execution in `core/runner.py`:
```python
loaded_modules, module_discovery_errors = load_module_descriptors(modules_path, config, package_prefix)
```
The module loader then discovers modules from three sources (combined and deduplicated):
1. **Filesystem directory** — scanned via `--path` (default: `modules/`).
2. **`plugin_packages.modules`** — Python packages listed in `config.yaml`.
3. **`hydrogen.modules` entry points** — registered by installed Python packages.
```python
def discover_modules(path: Path, config: Config, package_prefix: str = "modules") -> list[str]:
# 1. Scan filesystem
for entry in path.iterdir():
if entry.is_dir() and not entry.name.startswith("__") and (entry / "__init__.py").exists():
package_names.append(f"{package_prefix}.{entry.name}")
# 2. From config
package_names.extend(config.plugin_packages.modules)
# 3. From entry points
for ep in entry_points(group="hydrogen.modules"):
package_names.append(ep.value.partition(":")[0])
return list(dict.fromkeys(package_names)) # deduplicate
```
Filesystem rules:
- Hydrogen only inspects direct children of the modules directory.
- Entries must be directories.
- Directory names starting with `__` are skipped.
- The directory must contain `__init__.py`.
- The package is imported as `<package_prefix>.<directory_name>`.
Hydrogen does **not** recursively discover module packages inside nested directories. A module must be a first-level package under the selected modules root.
## Required Module Contract
Each Hydrogen module package must provide three things:
- `MANIFEST` — a `ModuleManifest` instance.
- `build_worker(config)` — a callable that returns a `BaseWorker`.
- `CONFIG_MODEL` (optional, recommended) — a Pydantic `BaseModel` subclass for config validation.
If either `MANIFEST` or `build_worker` is missing, the loader logs a warning and skips the module.
### `MANIFEST`
`MANIFEST` is validated against `core.schemas.manifest.ModuleManifest`.
```python
class ModuleManifest(BaseModel):
model_config = ConfigDict(extra="forbid", frozen=True)
identifier: str = Field(pattern=r"^[a-z][a-z0-9_-]*$")
name: str = Field(min_length=1)
category: str = Field(min_length=1)
version: str = Field(pattern=r"^\d+\.\d+\.\d+$")
api_version: str = Field("1", pattern=r"^\d+$")
description: str = ""
```
Required fields:
- `identifier` — must match `^[a-z][a-z0-9_-]*$`
- `name` — human-readable name, non-empty
- `category` — group name used for exclusion, non-empty
- `version` — semantic version `X.Y.Z`
- `api_version` — must match Hydrogen's `HYDROGEN_API_VERSION` (currently `"1"`)
`description` is optional and defaults to an empty string.
Example from the built-in SSH module:
```python
MANIFEST = ModuleManifest(
identifier="ssh",
name="SSH Security Audit",
category="ssh",
version="0.1.0",
api_version="1",
description="Audits OpenSSH server configuration.",
)
```
**Important:** `api_version` must match Hydrogen's API version. If it does not match, the module is rejected with a `ValueError`.
### `CONFIG_MODEL`
Modules can export a `CONFIG_MODEL` — a Pydantic `BaseModel` subclass that defines the expected shape of the module's configuration section from `config.yaml`.
```python
# modules/my_check/config.py
from pydantic import BaseModel, Field
class MyCheckConfig(BaseModel):
enabled: bool = Field(True)
threshold: int = Field(5, ge=1)
```
```python
# modules/my_check/__init__.py
from .config import MyCheckConfig
CONFIG_MODEL = MyCheckConfig
```
When `CONFIG_MODEL` is defined, Hydrogen validates the module's config section from YAML against this model before passing it to `build_worker()`:
```python
module_config = config.modules.get(module.manifest.identifier, {})
validated_config = module.config_model.model_validate(module_config)
worker = build_worker(module, validated_config)
```
If `CONFIG_MODEL` is not defined, Hydrogen passes an `EmptyPluginConfig` instance (an empty Pydantic model).
### `build_worker(config)`
The loader expects a callable named `build_worker`.
Signature:
```python
def build_worker(config: MyCheckConfig) -> BaseWorker:
...
```
Rules:
- It must be callable.
- It receives the **validated** module-specific config (a `BaseModel` instance from `CONFIG_MODEL`).
- It must return an instance of `BaseWorker`.
If it returns anything else, Hydrogen logs a warning and skips the module.
## Worker Contract
All custom workers must inherit `core.base.BaseWorker`.
```python
class BaseWorker(ABC):
config_model: type[BaseModel] = EmptyPluginConfig
def __init__(self, config: BaseModel | dict[str, Any] | None = None) -> None:
self.raw_config = config
self.config: BaseModel | dict[str, Any]
if isinstance(config, BaseModel):
self.config = config.model_dump(mode="python")
else:
self.config = config or {}
@abstractmethod
def run(self) -> AuditResults:
...
```
Key points:
- `__init__` accepts either a `BaseModel` (preferred) or a raw `dict`.
- The config is stored as both `self.raw_config` (original form) and `self.config` (dict form for easy access).
- `config_model` is a class attribute that can be set on the worker for reference.
- `run()` must return `AuditResults`.
The built-in SSH module shows the recommended pattern:
```python
class SSHSecurityWorker(BaseWorker):
config_model = SSHModuleConfig
def __init__(self, config: SSHModuleConfig | None = None) -> None:
super().__init__(config)
self.module_config = config or SSHModuleConfig()
def run(self) -> AuditResults:
if not self.module_config.enabled:
return AuditResults(status=AuditStatus.SKIPPED, findings=[], risk_level=0.0)
...
```
## Required Result Shape
Your worker's `run()` method must return `core.schemas.results.AuditResults`.
```python
class AuditResults(BaseModel):
status: AuditStatus
findings: list[AuditFindings]
risk_level: float
```
### `status`
Allowed values from `AuditStatus`:
- `pass` — the audit ran and found no issues.
- `fail` — the audit ran and produced one or more findings.
- `skipped` — the audit was intentionally not applicable on the current system.
### `findings`
Each finding is an `AuditFindings` object:
```python
class AuditFindings(BaseModel):
name: str
description: str = ""
severity: AuditSeverity
```
Severity values:
- `low`
- `medium`
- `high`
- `critical`
These severities feed directly into Hydrogen's exit code calculation.
### `risk_level`
`risk_level` is a float from `0.0` to `1.0` indicating how dangerous the results are.
Practical guidance:
- Return `0.0` for clean results.
- Return a bounded value up to `1.0`.
- Keep the calculation deterministic so reports remain comparable over time.
## Full Module Layout
Here is the full structure Hydrogen expects:
```text
modules/
my_check/
__init__.py
config.py # optional, recommended
worker.py
```
### `modules/my_check/config.py`
```python
from pydantic import BaseModel, Field
class MyCheckConfig(BaseModel):
enabled: bool = Field(True)
must_be_enabled: bool = Field(False)
```
### `modules/my_check/__init__.py`
```python
from core.base import BaseWorker
from core.schemas import ModuleManifest
from .config import MyCheckConfig
from .worker import MyCheckWorker
MANIFEST = ModuleManifest(
identifier="my_check",
name="My Custom Check",
category="custom",
version="1.0.0",
api_version="1",
description="Checks a custom hardening rule.",
)
CONFIG_MODEL = MyCheckConfig
def build_worker(config: MyCheckConfig) -> BaseWorker:
return MyCheckWorker(config)
```
### `modules/my_check/worker.py`
```python
from core.base import BaseWorker
from core.schemas.results import AuditFindings, AuditResults
from core.schemas.status import AuditSeverity, AuditStatus
from .config import MyCheckConfig
class MyCheckWorker(BaseWorker):
config_model = MyCheckConfig
def __init__(self, config: MyCheckConfig | None = None) -> None:
super().__init__(config)
self.module_config = config or MyCheckConfig()
def run(self) -> AuditResults:
findings: list[AuditFindings] = []
if self.module_config.must_be_enabled is not True:
findings.append(
AuditFindings(
name="custom_rule_disabled",
description="The custom rule is not enabled.",
severity=AuditSeverity.HIGH,
)
)
return AuditResults(
status=AuditStatus.PASS if not findings else AuditStatus.FAIL,
findings=findings,
risk_level=0.0 if not findings else 0.3,
)
```
## Wiring Module Configuration
Hydrogen validates and passes module config by manifest identifier.
Given this manifest:
```python
MANIFEST = ModuleManifest(identifier="my_check", ...)
```
The YAML section must be:
```yaml
modules:
my_check:
must_be_enabled: true
```
Hydrogen does this:
```python
module_config = config.modules.get(module.manifest.identifier, {})
validated_config = module.config_model.model_validate(module_config)
worker = build_worker(module, validated_config)
```
Important details:
- The key is `my_check`, not the directory label shown to users in a report title.
- If the key is missing, your module receives an empty dict, which is validated against your `CONFIG_MODEL` (using defaults).
- Config validation is strict: if you define `CONFIG_MODEL`, extra fields in YAML that are not in the model will raise a validation error (unless `extra="allow"` is set).
- If config validation fails, Hydrogen reports a `PluginRuntimeError` and continues with other modules — your worker is not created.
## Category-Based Exclusion
Hydrogen can disable modules by manifest category.
Example:
```yaml
exclude_categories:
- custom
```
If your module's manifest has `category="custom"`, Hydrogen skips it before worker creation.
This is useful for:
- Experimental checks.
- Platform-specific checks.
- Expensive checks that should be disabled in some environments.
## Module Discovery via Entry Points
Third-party modules can register themselves via Python entry points in the `hydrogen.modules` group.
Example `pyproject.toml`:
```toml
[project.entry-points."hydrogen.modules"]
my_check = "mycompany.hydrogen_modules.my_check"
```
The entry point value must be a Python package path (module or package). Hydrogen imports it and looks for `MANIFEST` and `build_worker` inside.
## Import Mechanics and Packaging Rules
Hydrogen imports modules using Python package names, not only filesystem paths.
For a module directory named `my_check` and default package prefix `modules`, Hydrogen imports:
```python
import modules.my_check
```
Therefore:
- The module directory must be importable as a Python package.
- Its `__init__.py` is the integration entry point.
- Import-time exceptions are not swallowed — they propagate and fail the module.
For third-party packages installed via pip, set `plugin_packages.modules` in config:
```yaml
plugin_packages:
modules:
- mycompany.hydrogen_modules
```
Or use entry points as described above.
## Behavior on Loader Errors
Hydrogen is tolerant of contract violations, but only up to a point.
**Handled by warning and skip:**
- Missing `MANIFEST`.
- Missing `build_worker`.
- Non-callable `build_worker`.
- `build_worker()` returning something that is not a `BaseWorker`.
**Handled by error reporting (module skipped, run continues):**
- Import failures while importing the package (caught in `load_modules()`).
- Config validation errors in `resolve_config()`.
- Runtime exceptions inside `worker.run()` (caught in `_run_single_worker()`).
**Not handled silently (can fail the entire run):**
- Manifest validation failures (e.g., `api_version` mismatch).
- Missing `CONFIG_MODEL` export with invalid type.
## Duplicate Module Detection
Hydrogen detects duplicate module identifiers during config resolution:
```python
if module.manifest.identifier in seen_module_ids:
runtime_errors.append(
PluginRuntimeError(
plugin_kind="module",
plugin_name=module.manifest.identifier,
stage="discovery",
message="duplicate module identifier",
)
)
continue
```
If two sources provide the same module identifier, the second one is rejected with an error.
## Recommended Development Pattern
For custom Hydrogen modules, the safest pattern is:
1. Define a `CONFIG_MODEL` Pydantic model for type-safe config validation.
2. Keep `__init__.py` small — just `MANIFEST`, `CONFIG_MODEL`, and `build_worker()`.
3. Put most logic in a separate worker module.
4. Set `config_model` on your worker class for consistency.
5. Use stable, machine-friendly finding names.
6. Return `skipped` when the check is not applicable on the current platform.
7. Use `api_version="1"` (match Hydrogen's `HYDROGEN_API_VERSION`).
## Example: Running a Custom Module Tree
If your module packages live outside the default `modules/` directory, you must align the filesystem path and package prefix.
Example:
```bash
python main.py --path custom_checks --package-prefix custom_checks
```
Then Hydrogen expects packages like:
```text
custom_checks/
__init__.py
my_check/
__init__.py
config.py
worker.py
```
Alternatively, add your package to `plugin_packages.modules`:
```yaml
plugin_packages:
modules:
- custom_checks.my_check
```
## Testing Checklist for a New Module
Before relying on a new Hydrogen module, verify all of the following:
1. The package imports cleanly.
2. `MANIFEST` validates successfully (especially `api_version`).
3. `CONFIG_MODEL` (if defined) validates correctly.
4. `build_worker()` returns a real `BaseWorker` instance.
5. `run()` always returns `AuditResults`.
6. All findings use valid `AuditSeverity` values.
7. The module behaves correctly when its config section is missing.
8. The module behaves correctly when it is excluded by category.
9. The module handles runtime errors gracefully (Hydrogen wraps exceptions).
## Summary
Hydrogen custom security modules are intentionally simple:
- One package per module.
- One manifest with `api_version` matching Hydrogen's.
- Optional `CONFIG_MODEL` for validated config.
- One worker builder receiving a validated config model.
- One worker object that returns structured results.
The most important implementation details are that module discovery supports three sources (filesystem, config packages, entry points), module config is validated by `CONFIG_MODEL` and keyed by `MANIFEST.identifier`, and import-time failures are reported but do not stop the entire run.